Adaptive Metropolis Sampling with Product Distributions
نویسندگان
چکیده
The Metropolis-Hastings (MH) algorithm is a way to sample a provided target distribution π(x). It works by repeatedly sampling a separate proposal distribution T (x, x) to generate a random walk {x(t)}. We consider a modification of the MH algorithm in which T is dynamically updated during the walk. The update at time t uses the {x(t < t)} to estimate the product distribution that has the least Kullback-Leibler distance to π. That estimate is the information-theoretically optimal mean-field approximation to π. We demonstrate through computer experiments that our algorithm produces samples that are superior to those of the conventional MH algorithm.
منابع مشابه
Covariance-Adaptive Slice Sampling
We describe two slice sampling methods for taking multivariate steps using the crumb framework. These methods use the gradients at rejected proposals to adapt to the local curvature of the log-density surface, a technique that can produce much better proposals when parameters are highly correlated. We evaluate our methods on four distributions and compare their performance to that of a non-adap...
متن کاملAdaptive Metropolis Sampling and Optimization with Product Distributions
The Metropolis-Hastings (MH) algorithm is a way to IID sample a provided target distribution π(x). It works by repeatedly sampling a separate proposal distribution T (x, x′) to generate a random walk {x(t)} which converges to a set of samples of π. Here, we introduce a T -updating phase after the cooling period and before sampling begins. In the updating phase, {x(t)} is used to update T at t a...
متن کاملA Class of Adaptive Importance Sampling Weighted EM Algorithms for Efficient and Robust Posterior and Predictive Simulation
A class of adaptive sampling methods is introduced for efficient posterior and predictive simulation. The proposed methods are robust in the sense that they can handle target distributions that exhibit non-elliptical shapes such as multimodality and skewness. The basic method makes use of sequences of importance weighted Expectation Maximization steps in order to efficiently construct a mixture...
متن کاملAdaptive Mixture of Student-t Distributions as a Flexible Candidate Distribution for Efficient Simulation
This paper presents the R package AdMit which provides flexible functions to approximate a certain target distribution and it provides an efficient sample of random draws from it, given only a kernel of the target density function. The core algorithm consists of the function AdMit which fits an adaptive mixture of Student-t distributions to the density of interest via its kernel function. Then,...
متن کاملLearning Stochastic Inverses for Adaptive Inference in Probabilistic Programs
We describe an algorithm for adaptive inference in probabilistic programs. During sampling, the algorithm accumulates information about the local probability distributions that compose the program’s overall distribution. We use this information to construct targeted samples: given a value for an intermediate expression, we stochastically invert each of the steps giving rise to this value, sampl...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2004